This tutorial written by Oska Fentem with David Reinstein
This appendix provides a brief introduction to the several types of software and processes used to creating websites such as Increasing Effective Charitable Giving and Researching and writing for Economics students. We aim to encourage others to participate in this collaborative work, and to spin off their own projects. If you would like to provide feedback or ask a question about these projects then using ‘hypothes.is’ is an easy way to do so.
This site (web-book project) is
Hosted:
The content is:
This relies heavily on:
To build this, we chose to use tools and software including:
We first give a brief overview of R & RStudio, Git & Github, and R Markdown & Bookdown, linking more extensive further resources/tutorials.
Getting a Github account should take about XXX minutes.
Here’s a guide to exactly how to do it.Installing Git and the GitHub Desktop should take about XXX minutes.
Here’s a guide to exactly how to do it.Git and GitHub can be a bit confusing. Here are some things that I wish I had known, that took my a while to figure out (unfold)
Git and Github are not the same thing … (explain)
A ‘commit’ does not actually change the files in the shared (remote) Github repo; you need to ‘push’ to do that
After ‘pulling’ from the remote repo, you may need to merge changes… (explain)
You can have several different ‘branches’ of the same Repo existing at the same time. When you switch to a new ‘branch’ the files you see on your computer will instantly and amazingly change to exactly the files in that branch. But don’t worry, the old branch is not lost.
… add some more
A brief overview of key functions inside Git (assuming a remote Github repo) including commits, pushes & pulls, forks & branches and pull requests: (unfold)
If you are familiar with Python, these R packages are roughly comparable to Python’s modules.
Installing R should take about XXX minutes.
Here’s a guide to exactly how to do it.Installing and configuring RStudio should take about XXX minutes.
Here’s a guide to exactly how to do it.Markdown is a popular set of formats (really a ‘syntax for specifying output’) for generating and authoring documents. The Rmarkdown format (rmarkdown package) is one flavor of Markdown that works with R to enable ‘dynamic documents’ involving text, data-analysis, and other elements. It can then export your work to a variety of outputs such as html, pdf and word documents. As well as this it can also be used to create webpages, such as the one you are currently reading. The power of Markdown files comes from the way that they are able include/embed code as well as data and tables, which is useful for writing reproducible research and creating websites.
The Bookdown package was built on the Rmarkdown package, but it adds many features to enable larger and more structured output, particularly ‘web books’ and web sites. As we use it, this these books combine multiple Rmarkdown files, with each such ‘Rmd’ file becoming its own HTML page.
Look at the list of headings on the left of this page: each second-level header is it’s own web-page (a distinct html link). “All the content in one scrolled page” is limited to a single first-level header.
docs: html output put here for web hosting
Folder: writing_econ_book
The Markdown format offers a simple plain-text notation for specifying the elements of documents, reports, web sites, etc. (It is much simpler and easier to read than html, latex, etc.) It is widely used by programmers, on comment boards/forums, and throughout the internet. For example, GitHub.com automatically renders markdown code, particularly in readme.md files.
Actually there are several varieties of markdown, but they mainly share key elements.
Markdown documents are usually saved as plain text files with the extension .md, e.g., report.md. These allow for an easy way to create a variety of outputs, particularly reports and text-focused web pages. The markdown format is converted into other formats (html, latex, etc.) with a variety of tools, particularly something called Pandoc.
What is Pandoc?
Pandoc is a tool (a program) for converting from one document format to another. It is incredibly powerful. The great thing about a format like markdown, or r-markdown, is that it is simple to write and peruse, and, with the help of Pandoc, it can convert into many many other useful formats for web pages, documents, presentations, etc.
Pandoc is built into other tools including the RMarkdown package (see discussion on Stackexchange here).
You can also install and use Pandoc directly in the command line, or try it out (in a limited but still useful way) on the web here
For more on Pandoc visit pandoc.orgIn the R (statistically focused) language there are tools such as knitR that allow R users to produce reports combining text, statistical output, and interactive content. These are generally written in “R-markdown” documents, saved as .Rmd rather than .md files. The R-studio interface, and several “add-ins”, also help facilitate this. This interface is very useful; in fact, it may be convenient to build web books and other content using this even if you are not planning to extensively integrate R code and data. (As in the present book, although I’m hoping to build this in).
Using R-markdown and Knitr (and other tools and add-ins like ‘Bookdown’) content from multiple sources can easily be embedded into these documents allowing users to easily display objects such as plots or regression output.
Text can be made italic using single asterisks *italic or bold by using asterisks **bold**.
Hashtags/pound signs (#) specify headers and subheaders, e.g., this third-level subsection header was created with the code:
### Some simple markdown rules {#simple-md-rules}
Where the bit in the curly braces allows us to link-back with the code [link back text whatever](#simple-md-rules) … rendering as link back text whatever.
Other key features are ordered lists and unordered lists:
- unordered first entry
- unordered second entry
- subelement of second entry
While basic markdown has a limited set of rules, there are many more formatting and content options for documents produced in (R)-Markdown, far too many to detail here. These may combine markdown code, html code, latex code, and more. The following cheatsheets are very useful for writing (R)-markdown documents:
Markdown documents allow for an easy way to write reports. Content from multiple sources can easily be embedded into these documents allowing users to easily display objects such as plots or tables of data.
Text can be made italic using single asterisks *italic* and bold by using double asterisks **bold**.
There are various text formatting options in Markdown, far too many to detail here… The following cheatsheets are very useful for writing markdown documents:
See also (most useful, but highly detailed):
Code chunks provide an easy way to embed code into your R Markdown files. The code language is not just limited to R either, as other languages can be used. This means that there is a wide variety of content which can be displayed in a chunk. Such as tables of data:
## Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1 5.1 3.5 1.4 0.2 setosa
## 2 4.9 3.0 1.4 0.2 setosa
## 3 4.7 3.2 1.3 0.2 setosa
## 4 4.6 3.1 1.5 0.2 setosa
## 5 5.0 3.6 1.4 0.2 setosa
## 6 5.4 3.9 1.7 0.4 setosa
Code chunks are defined by wrapping text inside ``` ```. The above example was coded using:
```{r}
head(iris)
```
Options can be specified inside of the curly brackets {} More information is provided here
Inline code is a quick and easy way to put snippets of R code. As an alternative to using code chunks, R code can simply be placed inside of `r `. For example, this can be used as an easy way to insert the value of a variable into a paragraph without inserting a chunk.
R Markdown also can make use of the LaTeX document preparation system, which is popular for writing technical documents with mathematical content. This allows us to publish documents which include equations such as:
\[y = \beta_0+\beta_1x_1 +\beta_2x_2+...+\beta_kx_k+u\]
Which is written using $$y = \beta_0+\beta_1x_1 +\beta_2x_2+...+\beta_kx_k+u.$$. Using $$ means that the equation will be centered on the page. Alternatively $ can be used in the same way, without the centering.
Bookdown allows for users to build their own custom styles in order to change the appearance of documents. To create styles for HTML projects a custom css file is used. For these projects, styles are contained in support/tufte_plus.css. To use a defined style, the user can specify options at the start of a chunk, or using a HTML wrapper as show below for margin notes.
A very useful guide to maths in R Markdown provides a detailed outline of the various mathematical symbols which can be used.
Bookdown allows for users to build their own custom styles in order to change the appearance of documents. To create styles for HTML projects a custom css file is used. For these projects, styles are contained in tufte_plus.css. To use a defined style, the user can specify options at the start of a chunk, or using a HTML wrapper as shown below for margin notes.
More on creating styles here. There are 3 main custom styles which are used throughout the projects:
{block2, type ='note'} can be specified at the start of the block, or a HTML wrapper can be used. This assigns the .note formatting from the tufte_plus.css file to the chunk.
The margin notes used in this project are inspired by the Tufte handout style developed by American statistician Edward Tufte.
<div class="marginnote">
Your margin note goes here.
</div>
Or margin notes can be added by using chunk options.
{block2, type='fold'} at the start of a chunk, or using a HTML wrapper.
As with any academic work, it is always important to reference sourced material. Across these projects the following software is used:
Pandoc provides a way to generate formatted references as well as a bibliography in Markdown.
(This is done through Pandoc tools.)
Oska, are you sure about this?
The bibliography file to be sourced is specified within ‘YAML’ content, which guides the processing of these documents. (YAML content is generally enclosed with a three-dash --- break at top and bottom.)
I generally specify the bibliography source in the YAML at the top of the .Rmd file, or for Bookdown project,s in the the YAML content in index.Rmd. (Y
@Oska – we should try to ecplain this yaml stuff a bit better.
The BibTeX format refers to a stylized file format which is used predomoninantly for lists of references, mainly and originally for working with latex.. BibTeX bibliographies use the .bib extension. For example the bibliography for this project is giving_keywords.bib. For more information on BibTeX see here
The Citr package provides functions to search Zotero and BibTeX libraries in order to insert references into Markdown files. Citr also features a plugin for RStudio which makes the referencing process even easier. Instructions for download, as well as a demonstration of the Rstudio plugin are provided here.
Zotero is a free open source reference manager, which enables users to sync their library of references across multiple devices. Similarly to other reference managers, Zotero offers plugins for popular browsers such as Chrome and Safari. Download Zotero
Better BibTeX for Zotero is a add-on for Zotero. Among other things it allows the Zotero library to be exported from Zotero for use in Markdown. Installation instructions are provided here.
I currently (25 Apr 2020) am having Zotero automatically maintain/output the key .bib file to a dropbox folder. Each project has code to routinely download this file with a command such as
download.file(url = "https://www.dropbox.com/s/3i8bjrgo8u08v5w/reinstein_bibtex.bib?raw=1", destfile = "reinstein_bibtex.bib")
One way is within RStudio
Be sure Github repo is synced so all files are present
Packages need to be installed, but this should (?) be done automatically when you build via the source(here("code", "baseoptions.R")) line in index.Rmd
Click ‘Build’, ‘Build all’, or the shortcut key shift-cmd-b
rmarkdown::render_site(encoding = 'UTF-8')Building may take some time, depending on how much code is present in the Rmd files and what that code does
It puts all the Rmd files specified in the _bookdown.yml into a single file, here labeled barriers-to-effective-giving.knit.md (I think), and then tuns that into html, also invoking bibtex along the way
Depending on your RStudio settings (-Tools, -Project Options, -Build tools, -Preview book after building), it may put up a ‘preview version’ of the site
All the ‘new’ output is directed to be put in the ‘docs’ folder, a bunch of html files. You can view those ‘local’ files in any web browser
Once you commit and push, the ‘new’ bookdown website should be up on the WWW
Get a Github account, contact daaronr AT gmail.com and tell him your github account ID (or the email you used to join should probably work as well)
Remember to ‘accept’ the invitation to the repos (here, the EA_giving_barriers repo; and possibly some other supporting repos as well). You should receive this invitations via email and it should also be in your “notifications” on Github.
As noted, GitHub is a web page and interface that acts as an external server and storage space for git projects/repos. It works well with this and also incorporate several additional features. You can see it and even interact with much of a repo simply via the GitHub webpage without even installing Git (but I strongly recommend that you do install Git as well as a tool like GitHub Desktop, unless you want to solely rely on command line Git).
In your account when you click on the repo you’ll see something like the screen above. There are many tabs, starting with the code tab. At the top of this, you will see the list of folders and files, with messages describing the latest comments.
Note here we also see:
48 ‘commits’
2 ‘branches’
2 contributors
Below this, some options allowing you to switch branch, manually upload files, clone or download etc.
Once you’ve installed Git you will want to ‘clone this repo’ to have it on your machine and to be able to easily work with it and commit and push and pull changes. You will do this clone either via the web site, GitHub Desktop or another application, or using the command line
Below the list of files you should see the “readme” for this repository. This is a file ‘README.md’ stored in the root directory of this repository. If you click on it or look at the file you’ll see it is written in markdown syntax but the GitHub website renders it into a nice format.
I typically use this readme to explain what the project is about and describe (and link) the folder structure.
Showing the most recent commits
Above, this shows the most recent commits.
Clicking on one of these commits will show you ‘what changed’ and old versus new versions.
Showing the most recent commits
For example, above we see something like a “split diff” view, with the ‘old version’ (before this commit) on the left and the ‘new version’ on the right. What is new is in green (with a ‘+’), and what is removed is in red highlight (with a “-”).
Here we see that …
in the file ‘sections/inertia.rmd’ a space has been added after ‘crowding out?’
in ‘present_puzzle.Rmd’ an (obsolete) ‘underline’ notation has been replaced with a third level markdown header (three # marks)
This is but one way to view and consider changes. Various text editors such as Atom and ViM also offer great tools, as does the Git program itself and the GitHub desktop application.
One way to ask questions, comment on changes and let people know about changes you made, is via adding a comment within a commit itself. (Check: can this be tied to an ‘issue’?)
Commenting on a part of a commit, notifying a collaborator
Above, we see that by clicking on a plus sign that appears just to the right of the the line number when viewing a commit, we can add a comment on that particular part of the commit. We can then flag another collaborator (see ‘@daaronr’ above … when you type the ‘@’ you get a dropdown of collaborators) who will be notified of this.
This mode of commenting and conversation has the advantage of avoiding cluttering up the actual code and text with excess comments.
You can also link each comment to an ‘issue’ (issues are discussed below) by adding a hash to the comment and citing the issue number. This comment and link to the place in the code or text for that commit will then show up when you look at that issue. This makes the discussion more organized, at least we hope.
Link an issue in a comment
the Project is a ‘Kanban board’ for managing tasks, responsibilities and progress
these should be entered as ‘issues’, enabling assignments and further discussion within the ‘issues’ pages
a github ‘Project’
Kanban for the github ‘Project’
One task/issue in the Kanban
Viewing this issue and its discussion
This project is closely connected to innovationsinfundraising.org. Much of these projects overlap, and there is a shared ‘database’ stored as an airtable Giving researchers shared
We had an earlier … tutorial on using the Airtable and Innovationsinfundraising.org here
I add a few more points below, more relevant to the current project:
Airtable is a collaborative web-based software with a variety of displays and organizational structures; it has many features of a relational database, and even more features if one engages their API. It is user-friendly, with a gui resembling a spreadsheet, and easy tutorials, instructions and examples. You can operate it from a browser or a web-driven app.
Each Airtable user can have any number of Bases, and bases can be shared in work groups.
“key_papers”; the papers providing the most relevant and strongest evidence for the tool, and “secondary papers”.
The “Categories” table provides and explains a number of “schema” we use to characterize both the tools and the “Barriers to effective giving” (discussed later).
categories table
Key papers are stored and organised in the ‘papers_mass’ table. This is crosslinked in several other tables. Within each paper ‘row’ there is a variety of relevant information and discussion on each paper.
key papers
The fundingwiki app automatically populates and updates information on the number of times each paper has been cited, using the Crossref database. Tool such as these will enable this to be a perrennial resource, rather than a frozen-in-time evaluation.
citations auto update
The table “EAlit_sections” outlines the (earlier?) structure of the EA barriers paper, already providing links to information that will be integrated.
organizing ealitpaper
This table also links directly to the papers_mass table, organizing the papers we are referencing and reviewing in each section.
organizing EAlit papers
The separate “Barriers to EAG” table is below.
This organizes and assembles the discussion and evidence on potential factors and categories of factors that may explain the limited amount of “effective giving”. This represents the largest part of our review paper; we focus on clear definitions of the most relevant psychological (and “behavioral economic”) biases, and carefully asses the available evidence. We focus specifically on evidence in the charitable domain, but we also consider the broader evidence for these biases in other contexts.
barriers to EAG
Again, this is older work, maybe already incorporated into the Bookdown?
For each barrier or bias, we consider why it is may be particularly relevant to effective giving.
barriers to EAG, why relevant
We further propose and discuss tools addressing these barriers and promoting effective charitable giving.
tools remedies
Like most things, when working with code the internet is your best friend. Listed below are several useful resources for learning about the material mentioned above:
Comments/notifications
In a variety of places within a repo when you are adding comments or content you can refer to a collaborator who will then receive a “notification” linking this content. (These are also called “callouts” in some systems.) These may come as as emails to that collaborator if they set a setting to get email notifications, but they will definitely appear as a notification, again that bell thing in the upper right hand corner.